Skip to main content

Best Practices

Data Preparation​

  • Always use the template procedure to create properly structured tables
  • Ensure your data tables include all required fields for the operation
  • Use appropriate data types that match Salesforce field definitions

API Selection Guidelines​

  • bulk: Invokes Bulk v1 API; best for large datasets (>30,000 records), supports parallel processing
  • bulkv2: Invokes Bulk v2 API; a modern alternative to Bulk v1 API, simpler controls
  • soap: Invokes SOAP API; best for smaller datasets (<30,000 records) or when you need precise control over system behavior
  • blob: For loading files
  • LockBuster: special data loading procedure; best for eliminating lock errors on junction tables and record volumes >100,000

Performance Tips (Loading)​

  • Use appropriate batch sizes for data loading. It is recommended to start with the largest allowable batch sizes and adjust for performance subsequently. These are also the Speediful defaults:
    • Bulk v1 API: 10,000 records per batch, parallel loading
    • Bulk v2 API: Automatically determined by Salesforce
    • SOAP API: 200 records per batch
  • Enable parallel processing for Bulk v1 operations when possible

The Bulk v1 API provides several options for tuning the performance of a load operation:

  • Enabling parallel processing will instruct Salesforce to processess batches simultaneously
  • Reducing the batch size will create more batches, which will allow Salesforce to parallelize more work. However, this will increase consumption of the 24-hr rolling 15,000 Bulk API batch limit
  • Extreme parallelism may be detrimental to overall throughput depending on what is required of Salesforce to process the records

Performance Tips (Query)​

  • Use PK Chunking with the Bulk v1 API for very large queries when supported (>200,000 records)

Error Handling​

  • All procedures throw error 883xx if the operation fails
  • Check the command output for detailed error messages
  • Results of data load operations are written back to your source table with success/error indicators

Security Recommendations​

  • Remove Salesforce connections that are no longer needed
  • Use appropriate Salesforce profiles and permission sets